System and method for high performance template driven metadata schema mapping and data storage for surveillance and sensor devices

ABSTRACT

A metadata database adaptor for use with a surveillance and/or sensor system is capable of adapting metadata messages of varying formats according to needs of metadata databases. In other aspects, a metadata data model template can allow a user to define a data definition that is used to map user defined data with the metadata data model template to form XML data schema. Template mapping knowledge can then be applied for fast XML data schema to non-XML database schema generation without translation processing. In yet other aspects, an application domain template can allow a user to define a query definition that can then be processed with a mapping of the application domain template to translate Xpath, Xquery, or others to a database query.

FIELD

The present disclosure generally relates to distributed surveillancesystems, and relates in particular to metadata schema mapping and datastorage for use with distributed surveillance systems.

BACKGROUND

The statements in this section merely provide background informationrelated to the present disclosure and may not constitute prior art.

Distributed video surveillance and sensor network systems usuallycontain varieties of devices and generate a great amount of media dataand metadata information continuously. To incorporate all these types ofequipments, store information, and retrieve data requires technologiesincluding data definition schema translation and mapping from XML toRelational, query transformation of XPath/XQuery to SQL, and multilevelcontent representation for complex data models.

The mechanisms used for data definition schema transformation includestructure mapping and model mapping. There are different methods beingproposed to solve the problems: Structure mapping, including “basic,shared & hybrid inlining technique”, “X-Ray”, “XML-DBMS”, and“Cost-based approach”; and Model mapping, including “Edge approach”,“Monet”, “Xrel”, and “XParent”. Each method has strengths to solve oneor more types of schema, but all have different advantages anddisadvantages.

SUMMARY

A metadata database adaptor for use with a surveillance and/or sensorsystem is capable of adapting metadata messages of varying formatsaccording to needs of metadata databases. In other aspects, a metadatadata model template can allow a user to define a data definition that isused to map user defined data with the metadata data model template toform XML data schema. Template mapping knowledge can then be applied forfast XML data schema to non-XML database schema generation withouttranslation processing. In yet other aspects, an application domaintemplate can allow a user to define a query definition that can then beprocessed with a mapping of the application domain template to translateXpath, Xquery, or others to a database query.

Further areas of applicability will become apparent from the descriptionprovided herein. It should be understood that the description andspecific examples are intended for purposes of illustration only and arenot intended to limit the scope of the present disclosure.

DRAWINGS

The drawings described herein are for illustration purposes only and arenot intended to limit the scope of the present disclosure in any way.

FIG. 1 is a block diagram illustrating an example distributedsurveillance system.

FIG. 2 is a flow diagram illustrating an example process flow forcollection creation.

FIG. 3 is a flow diagram illustrating an example process flow for querycriteria generation can also be template driven.

FIG. 4 is a block diagram illustrating an example data model templatefor a video surveillance system.

FIG. 5 is a block diagram illustrating an example application domaintemplate for a video surveillance system.

FIG. 6 is a block diagram illustrating a metadata database adaptorarchitecture.

DETAILED DESCRIPTION

The following description is merely exemplary in nature and is notintended to limit the present disclosure, application, or uses.

Distributed video surveillance and sensor network systems usuallyincorporate a variety of devices in terms of device types, vendors,communication protocols, and data specifications. Some functions thatcan be needed for such surveillance and sensor systems is to acquire,store, search, and retrieve data efficiently. The challenge is to befaster, more flexible, easier-to-use, and more accurate.

In general, new data models and operation flows need to be defined whena new surveillance and sensor network system is created. Most of thesurveillance and sensor devices and systems can communicate with othersusing XML data format although the data schema can be different, but areusually similar.

A template driven system can efficiently create and manage datadefinition schema as well as search and access operation querydefinitions for distributed surveillance and sensor network devices. Inorder to handle a variety of types and complex data formats, such asindustrial standard MPEG-7 and SensorML information, the systemframework takes advantages of using multi-modal schema transformationand provides open interfaces for plug-in of multiple schema mapping andquery transformation engines. It also provides multi-stage querypre-processing and post-processing for handling complex queries. Inaddition, it can be equipped with the capability to enhance templatecontents automatically via observing (mining) from user's operationhistorical data, creating indexes for data schema, and adding querystatements. Templates can include application domain data model, dataschema definition, query definition, index definitions, and datarendering mechanisms.

The data definition schema mapping mechanism in this system can allowmultiple methods to plug-in and select it by dynamic declaration. It istime consuming process to define, translate, and test for a correctschema for a new system. Using a template approach can reduce the effortof the process while avoiding errors, and provide more high performancesystem. Pre-defined and tested generic templates can be used fordifferent styles of schema and queries with proper indexes. When a newapplication data schema plan is being created, the system can takeinputs from an administrator and process them with template knowledgeand available plug-in methods to map to one or more predefined DB schemaformats without time-consuming processes. A similar concept is used forquery operations.

Since the system framework can have the capability to allow plug-in ofdifferent schema mapping and query transformation processor modules, itcan handle many styles of schema, from simple definitions of simpledevices to complex definitions like MPEG-7, and different querylanguages like XPath, XQuery, SQL, etc. to fit all the needs of videosurveillance and sensor network applications.

Referring now to FIG. 1, a configuration for surveillance and sensorapplication systems can include various components. However, thisconfiguration is merely an example. It is envisioned that many differentconfigurations of such systems can be employed. Some components of theconfiguration include video surveillance devices 100-102 and sensordevices 104-106. Also, applications 108-112 can include a surveillanceapplication 108, a surveillance and sensor application 110, and a sensornetwork application 112, each of which can have device metadata accessinterfaces, sensor and/or surveillance templates, and metadata databaseadaptor interfaces. Further, metadata messages 114-124 of varyingformats can be communicated by Ethernet 126 and stored in databases128-132. Still further, metadata database adaptors 134-138 can handlemessages of varying formats and adapt them according to the needs oftheir respective databases.

According to the example configuration, there are two types of MetadataDB Adaptors in the distributed system: Regular DB Adaptor, such asadaptors 134-136; and Master DB Adaptor, such as adaptor 138. Both typesof database adaptors can contain interfaces 140, service managers 142,functional modules 144, database managers 146, and database connectionpools 148. In one respect, the regular database adaptor and masterdatabase adaptor differ in that the service manager of the regulardatabase adaptor is an adaptor service manager, while the servicemanager of the master database adaptor is a master service manager. Inanother respect, the two types of adaptors differ in the types of theirfunctional modules. Specifically, both types of adaptors can have querymapping manager modules and data processing manager modules. However,the regular database adaptor can have a collection templates managermodule, a schema mapping manager module, and an index manager module. Incontrast, the master database adaptor can have an adaptor directorymodule, a collection schema directory manager module, and a collectiontemplate directory manager module.

Metadata databases 128 and 130 can differ from master metadata database132. In particular, databases 128 and 130 can each store collectioninformation, data tables, query statements, and index metadata. Incontrast, master metadata database 132 can store a template database, anadaptor directory, a collection directory, and rules and configurations.

The database management system (DBMS) can be any kind (relational,object, object-relational, file, etc.). SQL and relational DB schema areused as examples herein, but it should be readily understood that anykind of DBMS combined with the template approach can save effort andtime for users and for field deployment by reducing errors andfacilitating the deployment process.

The functional modules of regular metadata database adaptors 134-136 canhave specialized functions and, in some cases, be designed to interactwith modules of the master metadata database adaptor. For example, acollection template manager module can manage collection creation,modification, and deletion in a local metadata DB, and post collectionregistration to remote master DB adaptor 138. Also, a schema mappingmanager module can handle mapping, translation, and generation of datadefinition schema to XML schema and to database data schema likerelational DB schema. If the data definition schema is based on anexisting template, the process time can be reduced using pre-processedmapping stored in the template. Schema mapping manager module can alsocontain a shared memory module to store active schema mapping metadatafor other operation modules to use. It can reduce the re-processing timefor each operation, and provide high performance schema processing andgeneration. Further, an index manager module can provide DB table indexcreation and deletion management. It can also have an intelligent miningfeature to look into query operation historical data and learn fromquery pattern data and search operation time. It can determine if a newindex needs to be created for enhancing the search performance when thehistorical data of search operation time are over a pre-definedthreshold.

Modules common to both types of adaptors can provide functionalitiesuseful to both types of modules. For example, a query mapping managercan handle mapping, translation, and generation of query criteria toXPath/XQuery and to database query languages like SQL. It can save agreat amount of time if the query criteria are based on the existingtemplate. Query manager module can also have an intelligent miningfeature to look into query operation historical data and learn fromquery pattern data to determine if a new query template is needed togenerate query criteria.

Functional modules unique to the master database adaptor can provideseveral directory services. For example, these modules can include anadaptor directory, a collection template directory, a schema directory,a query template directory, etc.

Turning now to FIG. 2, an example process flow for collection creationcan be template driven. At step 200, a collection can be created for anapplication system, such as a video surveillance system. At step 202, aproper predefined template can be searched for and selected. If there isno proper predefined template at 204, then the user can create a newSurveillance/Sensor data definition XML schema at step 206. A dynamicschema translation and generation process at step 208 produces templatemapping and data rules 210. However, if there is a proper template at204, then the user can define a surveillance/Sensor data definition viaa template GUI at step 212 then a user-defined data definition ispopulated at 214 and used at step 216 to map the user defined data withthe template to form XML data schema. Then, at step 218, it is possibleto apply template mapping knowledge for fast XML data schema to DBschema (e.g., relational DB schema) generation without translationprocessing. Then, at step 220, the collection can be registered andpopulated with metadata and data tables 222 in the DB.

Turning now to FIG. 3, an example process flow for query criteriageneration can also be template driven. Starting at step 300, anexisting collection of the application system can be loaded. A searchfor a predefined query template can be conducted at 302 and a templatecan be selected. Then, a user can define a query definition at step 304via a template GUI. A populated user-defined query definition 306 canthen be processed at step 308 with the query template mapping andtranslate/generate Xpath or Xquery. At step 310, template mappingknowledge 312, including template mapping data and rules, can be appliedfor fast Xpath/Xquery to DB query (i.e., SQL) generation. Then, thequery can be saved at step 314 into collection query metadata in the DB,such as query lists metadata and data table 316.

Turning now to FIG. 4, an example data model template for a videosurveillance system can provide mapping data and rules for describingdata. For example, a surveillance application domain 400 can includeapplication domain collections 402, user navigation and preferences 404,and summary and view 406. Also, event handling metadata 408 can includerule definition 410, event mapping 412, device mapping 414, and actionmapping 416. Further, configuration metadata 418 can include device 420,device group 422, site route 424, control and play API 426, action 428,and SNMP/MIB 430. Yet further, AV media metadata 432 can include AVmetadata information 434, content description 436, event 438, AV-eventmapping 440, and AV thumbnail information 442 (i.e., image, sound andvideo). Further still, metadata management 444 can include schemadefinition 446, operation module definition 448, schema dictionary 450,and directory 452. Finally, basic elements 454 can include databaseconnection configuration 456, XML-RDB mapping definition 458, basic datatype 460, exception handling 462, and log 464.

Turning now to FIG. 5, an application domain template can providemapping data and rules for describing an application domain. Forexample, Surveillance and Sensor_UniversalXML 500 can include videosurveillance applications 502, secure access control application 504,sensor network management applications 506, and other applications 508.Also, secure access control applications can include AM 510, device 512,AV 514, and OAM 516. Further, video surveillance applications 502 caninclude Action Management (AM) 518, device 520, AV media metadata 522,and OAM 524. Yet further, AM 518 can include AM info 526, device mapping528, and action mapping 530. Further still, device 520 can includedevice info 532, device group 534, device control and play API 536,device action 538, device model 540, and zone 542. Yet even further, AVmedia metadata can include content 544, event 546, description 548,EventContent_Relation 550, and Video_Thumbnail_metadata 552. Yet furtherstill, OAM 524 can include User_Management 560, Role_Definition 562,XML_Schema 564, DB_Adaptor_Configuration 566, XML-RDB_Mapping_Rule 568,and XML-RDB_Module_Registration 570.

AV media metadata 522 can be especially rich. For example, content 544can include video 572, sensor content 574, audio 576, other content 578,and image 580. Also, video 572 can include media occurrence 582,temporal segment 584, spatial segment 586, and spatio temporal segment588, while each of segment 584-588 can include instances of description590-594. Further, event 546 can include ID 596, type 598, cause 600,source 602, date/time 604, status 606, UID 608, and action 610. Stillfurther, description 548 can include metadata 611, visual description612, tracking description 614, audio description 616, NEWDevice_Desc618, and object description 620, while EventContent_Relation 550 caninclude the relation 622. Finally, Video_Thumbnail_Metadata 552 caninclude ID 624, Image_Property 626, Image_URL 628, Content_ID 630, andSegment_ID 632.

In general, video surveillance systems produce and interact with videomedia data and video analysis data. MPEG-7 is one of the industrystandards for describing multimedia information of media includingvideo, and it is used by many video analysis and recognition systems todescribe the media information. Due to the complexity of MPEG-7 datadefinition schema, designing an efficient MPEG-7 information datarepository system is a challenge.

Turning now to FIG. 6, an architecture of a metadata database adaptorcan have various components. For example, it can have metadata databaseadaptor interfaces 140 and a service manager 142. It can also have acollection/template manager module 650 that creates a collection at step652, performs mapping at step 654, performs collection registration atstep 656. Module 650 can contain functions 658-662 for gettingcollection names, getting collection information, and deletingcollections. Further, metadata schema mapping manager module 664 canhave an Xschema creator 666, a relational schema processor 668, and X2RMapping 670 for mapping between the Xschema and relational schema.Xschema creator 666 can have an XML schema processor 672 and an indexannotation processor 674. Still further, it can have utilities modules676 that can include simple path 678, ID service 680, RDBLog 682, andStat 684. Yet further, database manager module 686 can have an indexcreator 688 with DB Inti, DB Connection, Table, and Field capabilities,A DB manager 690 for executing DB operations, database connection 692,database connection pool 694, and database connection driver 696. Evenfurther, module 664 can have a share memory management module 698 thatcan have instructions 700-704 for getting active collection information,getting an active collection structure, and threadsafe creation,getting, and deletion of an active collection in an active memory cache.Finally, database management system 706 can include a metadatarepository 708 and a persistent collection template management database710. Metadata repository 712 can have: (a) a collection template 712 ofelements; (b) a face template 714 with channel number, data time, facefeature, thumbnail, and other information, (c) an event template 716with event ID, channel number, data time, event type, and otherinformation; (d) an MPEG7 device template 718; and (e) others. Templatemanagement database 710 can have: (a) collection metadata 720, includingcollection ID, collection information, and schema; (b) collection tablename 722, including collection ID, collection information, and tablemetadata; (c) document metadata 724, including collection ID, documentID, and document metadata; (d) index table 726, including collection IDand index; (e) PSTable 728, including collection ID, eid, ename, dp,ord, rtype, pid, chid, ltid, rtid, occ, tname, and fname; and XSTable730, including collection ID, XS object, and X2R object 730.

The systems and methods described above provide several advantages overprior systems and methods. For example, these systems and methodsprovide an easy and efficient field programmable platform to create andmanage metadata structure for DB storage. Also, template driven mappingcan reduce the difficulties and errors in creation of surveillance andsensor device metadata schema. Further a multi-modal mechanism in datadefinition schema transformation to DB storage can achieve highperformance access and retrieval. Yet further, multi-level indexing anda multi-stage query processes achieve both efficient search and accuratesearch needs. Further still, the systems and methods can easilyadd/remove new types of video surveillance devices or sensor deviceswith minimum metadata collection configuration for different applicationdomains. Finally, intelligent self-learning query discovery and miningmechanisms can be included for distributed surveillance and sensor data.

1. A metadata database adaptor for use with a surveillance and/or sensorsystem, comprising: an adaptor interface connecting with one or moreapplications and receiving metadata messages from the applications; ametadata database interface connecting with a metadata database; and oneor more functional modules adapting metadata messages of varying formatsaccording to needs of the metadata database, wherein at least one of themodules employs a metadata data model template that allows a user todefine a data definition that is used to map user defined data with themetadata data model template and form XML data schema so that templatemapping knowledge can then be applied for fast XML data schema tonon-XML database schema generation without translation processing, andwherein at least one of the modules employs an application domaintemplate to allow a user to define a query definition that can then beprocessed with a mapping of the application domain template to translatea query of a first query language to a database query of a second querylanguage.
 2. A surveillance and sensor application system, comprising:at least two devices including at least one of video surveillancedevices or sensor devices; at least two applications including at leastone of a surveillance application or a sensor network application,wherein the applications produce metadata messages of varying formats;at least two metadata databases storing the metadata messages of varyingformats; and at least two metadata database adaptors associated with themetadata databases and adapting the metadata messages of varying formatsaccording to the needs of their respective databases.
 3. The system ofclaim 2, wherein the adaptors include a regular metadata databaseadaptor and a master metadata database adaptor, wherein the regulardatabase adaptor contains a collection template manager module thatmanages collection creation, modification, and deletion in itsrespective metadata database, and posts collection registration to themaster metadata database adaptor.
 4. The system of claim 2, wherein thedatabase adaptors contain interfaces, service managers, functionalmodules, database managers, and database connection pools.
 5. The systemof claim 2, wherein at least one of the adaptors includes a schemamapping manager module that can handle mapping, translation, andgeneration of data definition schema to XML schema and to database dataschema, wherein the schema mapping manager can dynamically selectmultiple mapping and transformation engines based on a templatedefinition in order to take advantage of multi-modal mapping andtransformation engines.
 6. The system of claim 5, wherein the schemamapping manager module has a shared memory module to store active schemamapping metadata for other operation modules to use.
 7. The system ofclaim 2, wherein at least one of the adaptors includes an index managermodule that provides metadata database table index creation and deletionmanagement.
 8. The system of claim 7, wherein the index manager modulehas an intelligent mining feature to look into query operationhistorical data and learn from query pattern data and search operationtime.
 9. The system of claim 8, wherein the index manager moduledetermines if a new index needs to be created for enhancing searchperformance when the historical data of search operation time is over apre-defined threshold.
 10. The system of claim 2, wherein the adaptorsinclude a query mapping manager that handles mapping, translation, andgeneration of query criteria to the first query language and the secondquery language.
 11. The system of claim 10, wherein the query mappingmanager module has an intelligent mining feature to look into queryoperation historical data and learn from query pattern data to determineif a new query template is needed to generate query criteria.
 12. Thesystem of claim 2, wherein at least one of the adaptors includes one ormore directory services, including at least one of an adaptor directory,a collection template directory, a schema directory, or a query templatedirectory.
 13. A method for collection creation, comprising: obtainingan XML data schema; registering a collection; and using the XML dataschema to populate the collection with metadata in a database.
 14. Themethod of claim 13, further comprising attempting to search for andselect a proper predefined template.
 15. The method of claim 14, furthercomprising: if there is no proper predefined template, then permitting auser to create a new Surveillance/Sensor data definition XML schema; andperforming a dynamic schema translation and generation process toproduce template mapping and data rules.
 16. The method of claim 14,further comprising: if there is a proper template, then permitting auser to define a data definition via a graphic user interface of theproper template; populating a user-defined data definition; using thedata definition to map user defined data with the proper template toform the XML data schema; and applying template mapping knowledge forfast XML data schema to non-XML database schema generation withouttranslation processing.
 17. A method of query criteria generation,comprising: processing a query definition with a query template mappingto generate a query according to a first query language; applyingtemplate mapping knowledge, including template mapping data and rules,for fast generation, based on the query, of a database query accordingto a second query language; and saving the database query intocollection query metadata in a database.
 18. The method of claim 17,further comprising loading an existing collection of an applicationsystem.
 19. The method of claim 18, further comprising: searching forand selecting a predefined query template; and allowing a user to definethe query definition via a graphic user interface of the template. 20.The method of claim 17, wherein the first query language is at least oneof Xpath or Xquery, and the second query language is SQL.